tantivy-tokenizer-api 0.3.0

Tokenizer API of tantivy
Documentation

#Tokenizer-API

An API to interface a tokenizer with tantivy.

The API will be kept stable in order to not break support for existing tokenizers.